The present invention relates to management of trick playback of digital video.
Techniques for digital transmission of video promise increased flexibility, higher resolution, and better fidelity. Recent industry collaborations have brought digital video closer to reality; digital video transmission and storage standards have been generated, and consumer digital video products have begun to appear. The move toward digital video has been encouraged by the commercialization of digital technologies in general, such as personal computers and compact discs, both of which have increased consumer awareness of the possibilities of digital technology.
Personal computers, which have recently become common and inexpensive, contain much of the computing hardware needed to produce digital video, including a microprocessor/coprocessor for performing numeric calculations, input and output connections, and a large digital memory for storing and manipulating image data. Unfortunately, personal computers are not suitable for consumer digital video reception, because the microprocessor in a personal computer is a general purpose processor, and typically cannot perform the calculations needed for digital video fast enough to produce full-motion, high definition video output.
Accordingly, special purpose processors, particularly suited for performing digital video-related calculations, have been developed for use in digital video receivers for consumer applications. A specific processor of this kind is disclosed in commonly-assigned, copending U.S. patent application Ser. No. 08/865,749, entitled SPECIAL PURPOSE PROCESSOR FOR DIGITAL AUDIO/VIDEO DECODING, filed by Moshe Bublil et al. on May 30, 1997, which is hereby incorporated by reference herein in its entirety, and a memory controller for use therewith is disclosed in commonly-assigned, copending U.S. patent application Ser. No. 08/846,590, entitled xe2x80x9cMEMORY ADDRESS GENERATION FOR DIGITAL VIDEOxe2x80x9d, filed by Edward J. Paluch on Apr. 30, 1997, which is hereby incorporated herein in its entirety.
The above-referenced U.S. patent applications describe an application specific integrated circuit (ASIC) for performing digital video processing, which is controlled by a reduced instruction set CPU (RISC CPU). The RISC CPU controls computations and operations of other parts of the ASIC to provide digital video reception. As is typical of CPU""s of many varieties, the CPU described in the above-referenced U.S. patent applications supports flow control instructions such as BRANCH, CALL and RETURN, as well as providing hardware interrupt services.
Due to the limitations of the RISC CPU, a task and stack manager procedure is required to monitor task flags, prioritize task flags, manage subroutine calls (the hardware does not support nesting of subroutine calls), and provide virtual instruction memory management. A specific procedure of this kind is disclosed in commonly-assigned, copending U.S. patent application Ser. No. 08/866,419, entitled TASK AND STACK MANAGER FOR DIGITAL VIDEO DECODING, filed by Taner Ozcelik et al. on May 30, 1997, which is hereby incorporated by reference herein in its entirety.
However, even with task and stack management, the host processor would have to be intimately integrated with the ASIC in order to command it. For example, in order for the host to command the ASIC to perform complex operations, the host would have to generate a number of specific commands and also would have to prioritize and schedule each specific command and monitor its completion before submitting the next specific command. The host would also have to become involved in synchronizing some commands with events such as during vertical blanking or over a number of video frames.
In a typical early generation video decoding system, such as a DVD player, the manner in which video is decoded is relatively limited. Specifically, the host requests that one or more frames of video be decoded, into specific video buffers. The video buffer that will receive a decoded frame can be predetermined from.the playback mode and the kind of video frame that is being decoded (I or P reference frame or B frame). The host then supplies the corresponding encoded video, and when the decoding is done, the host causes display of the decoded video frames from predetermined video buffers.
If the decoding operation takes longer than the duration of display of a frame, then a frame must be repeatedly displayed until the decoding is complete. During normal forward playback, or slow forward playback, or slow reverse playback, one frame is decoded and displayed at a time following this process. Typically, during normal or slow forward playback, the decode of a frame is performed in a shorter time than the display of the prior frame, so the display can advance at the normal playback speed. For reverse playback, a frame prior to the current frame is decoded, and then the decoded frame is displayed repeatedly while the next prior frame is decoded. Since MPEG encoded data is intended to be forward rather than reverse decoded, the decoding of a prior frame can take a substantial period of time; therefore, reverse playback of all frames is of necessity substantially slower than normal forward playback. During fast forward or fast reverse playback, to enable fast playback, only certain frames (typically, the MPEG reference frames) are decoded and displayed using this process. For example, an MPEG group of pictures or video object unit is supplied to the decoder, and, e.g., three reference frames are decoded and buffered. Then, the decoded reference frames are displayed in forward order (for fast forward playback) or reverse order (for fast reverse playback), until the last reference frame is being displayed, at which time a next or previous MPEG GOP or VOBU, as appropriate, is supplied to the decoder and, again, the appropriate number of reference frames is decoded.
A difficulty with this approach is that, other than in the normal forward playback mode, the playback speed is uneven and can be unacceptably slow. For example, during reverse playback, the number of frames that must be decoded, to decode the prior frame, can be different based upon the relative position of the prior frame to the reference frames of the MPEG structure. Thus, the time taken to decode the prior frame can vary substantially, resulting in an uneven reverse display speed. Moreover, since multiple frames must be parsed and at least two reference frames must be decoded before each change of the display to a prior frame can be decoded, the speed of reverse playback is slow, often unacceptably so. Also, in the fast forward or reverse playback modes, it will be noted that the last frame of a set of reference frames may need to be displayed longer than the other reference frames from the same set, while the decoding of the next set of reference frames is being performed. This may lead to an uneven display speed as some frames remain on the screen longer than others. Also, the fastest forward or reverse playback speed may not be fast enough for some users.
In accordance with principles of the present invention, the foregoing difficulties are overcome by a single-chip application specific integrated circuit (ASIC) having its own central processing unit (CPU), which provides autonomous management of playback of digital video and audio within the ASIC. The CPU is programmed with instructions implementing a playback state machine. Commands received by the CPU from a host establish a current playback state for management of playback of digital video and audio by the ASIC. The CPU then responds to a video field synchronization signal and the current playback state, without external instruction, to determine whether to display digital video, whether to decode digital video for display, whether to repeat display of previously decoded digital video, and/or whether to skip over digital video prior to decoding digital video for output. In this manner, the host may deliver commands to the CPU and cause the ASIC to transition between playback states to provide desired playback of the digital video and audio, without low-level involvement in the details of playback of individual fields or frames.
The ASIC software includes routines performing management of the decode and display processes, and of the buffers utilized in each process. This permits the ASIC to perform decoding and display operations in an optimal manner for smooth and rapid playback in trick modes including fast forward, fast and slow reverse. By managing the buffers within the ASIC software, the use of the buffers and the scheduling of the decoding and display can be made substantially more sophisticated.
In the detailed embodiment of the invention described below, there is a STOP state in which the ASIC responds to the video field synchronization signal by not decoding or displaying any digital video. In a PLAY state, the ASIC responds to the video field synchronization signal by decoding and displaying a new video field. In a SLOW FORWARD state the ASIC responds to the video field synchronization signal by repeatedly displaying previously decoded digital video for a number of repetitions, then decoding and displaying new video fields. The number of repetitions to be performed in the SLOW FORWARD state are delivered as part of the command to enter the SLOW FORWARD state, thus providing an adjustable SLOW FORWARD playback speed. In a FAST FORWARD state, the ASIC responds to the video field synchronization signal by repeatedly displaying previously decoded digital video reference frames for a number of repetitions, then skipping over non-reference frames and decoding and displaying a new reference frame. The number of repetitions of a reference frame to be performed in the FAST FORWARD state, is identified in the FAST FORWARD command, as is a limit on the number of reference frames to be decoded from each video object unit. In a PAUSE state the ASIC responds to the video field synchronization signal by repeatedly displaying a previously decoded video frame. In a FAST REVERSE state the ASIC responds to the video field synchronization signal by repeatedly displaying previously decoded digital video reference frames for a number of repetitions, then skipping backwards over non-reference frames and decoding and displaying a new reference frame. The number of repetitions of a reference frame to be performed in the FAST REVERSE state, is identified in the FAST REVERSE command, as is a limit on the number of reference frames to be decoded from each video object unit. In a REVERSE DECODE state the ASIC responds by buffering decoded video prior to a currently displayed video frame, so that this decoded video can be output in a step reverse or slow reverse manner under command and control of the host. In this state, the ASIC is responsible for requesting encoded video data from a host, so that the ASIC can accurately and efficiently manage both the decoding and display processes.